The computation of divergence is studied. Covariance matrices to be analyzed admit a common diagonalization, or even triangulation. Sufficient conditions are given for such phenomena to take place, the arguments cover both real and complex matrices, and are not restricted to Hermotian or other special forms. Specifically, it is shown to be sufficient that the matrices in question commute in order to admit a common triangulation. Several results hold in the case that the matrices in question form a closed and bounded set, rather than only in the finite case.